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Abstract. We prove two basic conjectures on the distribution of the small- 
est singular value of random nxn matrices with independent entries. Under 
minimal moment assumptions, we show that the smallest singular value is 
of order n -1 / 2 , which is optimal for Gaussian matrices. Moreover, we give 
a optimal estimate on the tail probability. This comes as a consequence of 
a new and essentially sharp estimate in the Littlewood-Offord problem: for 
i.i.d. random variables Xk and real numbers ak, determine the probability 
p that the sum ^ fc cikXk lies near some number v. For arbitrary coefficients 
ak of the same order of magnitude, we show that they essentially lie in an 
arithmetic progression of length 1/p. 



1. Introduction 

1.1. Invertibility of random matrices. In this paper we solve two open 
problems on the distribution of the smallest singular value of random matrices. 

Let A be an n x n matrix with real or complex entries. The singular values 
Sk{A) of A are the eigenvalues of \A\ = V 'A* A arranged in the non- increasing 
order. Of particular significance are the largest and the smallest singular values 

Sl (A) = sup ||Ar|| 2 , s n (A) = inf \\Ax\\ 2 . 



=1 



=i 



These quantities can obviously be expressed in terms of the spectral norm - 
the operator norm of A considered as an operator on Indeed, Si(A) = \\A\\, 
and if the matrix A is non-singular then s n (A) = 1/||A _1 ||. The smallest 
singular value thus equals the distance from A to the set of singular matrices 
in the spectral norm. 
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The behavior of the largest singular value of random matrices A with i.i.d. 
entries is well studied. The weakest assumption for its regular behavior is 
boundedness of the fourth moment of the entries; then 

(1.1) s i{A) ~ 7-1 with high probability. 

Indeed, by [331 [1] the finite fourth moment is necessary and sufficient for 
si(A)/n l l 2 to have an almost sure limit as n — > oo, and this limit equals 2. 
Latala [H] showed that f 1 1.11) holds under the forth moment assumption even 
if entries are not identically distributed. 

Much less has been known about the behavior of the smallest singular value. 
In the classic work on numerical inversion of large matrices, von Neumann 
and his associates used random matrices to test their algorithms, and they 
speculated that 

(1.2) s n (A) ~ n^ 1 / 2 with high probability 

(see [32], pp. 14, 477, 555). In a more precise form, this estimate was conjec- 
tured by Smale [24J and proved by Edelman [5] and Szarek [28] for random 
Gaussian matrices A, those with i.i.d. standard normal entries. Edelman's 
theorem states that for every e > 

(1.3) P(s„(A) < en- 1/2 ) ~ e. 

Prediction fll.2p for general random matrices has been an open problem, 
unknown even for the random sign matrices A, those whose entries are ±1 
symmetric random variables. In this paper we prove the prediction fll.2p in 
full generality under the aforementioned fourth moment assumption. 

Theorem 1.1 (Invertibility: fourth moment). Let A be annxn matrix whose 
entries are independent real random variables with variances at least 1 and 
fourth moments bounded by B. Then, for every 5 > there exist e > and Uq 
which depend (polynomially) only on 5 and B, and such that 

P(snGA) < sn' l/2 ) < 5 for all n > n . 

This shows in particular that the median of s n (A) is of order n~ l l 2 . 

Under stronger moment assumptions, more is known about the distribution 
of the largest singular value, and similarly one hopes to know more about the 
smallest singular value. 

Indeed, Soshnikov [25J proved that the limiting distribution of S\(A) is pre- 
cisely the Tracy- Widom law for all matrices with i.i.d. subgaussian entries. 
Recall that a random variable £ is called subgaussian if its tail is dominated 
by that of the standard normal random variable: there exists B > such that 

(1.4) P(|f | > t) < 2 exp(-t 2 /B 2 ) for all t > 0. 
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The minimal B here is called the subgaussian momen fi of f . Inequality (Ojl 
is often equivalently formulated as a moment condition 

(1.5) (E|f| p ) 1/p < CBv^ forallp>l, 

where C is an absolute constant. The class of subgaussian random variables 
includes many random variables that arise naturally in applications, such as 
normal, symmetric ±1, and in general all bounded random variables. 

One might then expect that the estimate (11.31) for the distribution of the 
smallest singular value of Gaussian matrices should hold for all subgaussian 
matrices. Note however that (11. 3p fails for the random sign-matrices, since they 
are singular with positive probability. Estimating the singularity probability 
for random sign-matrices is a longstanding open problem. Even proving that 
it converges to as n — > oo is a nontrivial result due to Komlos |14j . Later 
Kahn, Komlos and Szemeredi [13J showed that it is exponentially small: 

(1.6) P (random sign matrix A is singular) < c n 

for some universal constant c G (0, 1). The often conjectured optimal value of 
c is 1/2 + o(l) [2UI [33], and the best known value 3/4 + o(l) is due to Tao and 

Vu [231130!. 

Spielman and Teng [2B] conjectured that (11.31) should hold for the random 
sign matrices up to an exponentially small term that accounts for their singu- 
larity probability: 

F(s n {A) < en- 1 ' 2 ) < e + c n . 

In this paper, we prove Spielman- Teng' s conjecture for all matrices with 
subgaussian i.i.d. entries, and up to a constant factor which depends only on 
the subgaussian moment. 

Theorem 1.2 (Invertibility: subgaussian). Let £ 1; . . . , £ n be independent cen- 
tered real random variables with variances at least 1 and subgaussian moments 
bounded by B . Let A be an n x n matrix whose rows are independent copies of 
the random vector (£i, . . . , £ n ). Then for every e > one has 

(1.7) P(s„(A) < en- 1 ' 2 ) < Ce + c n , 

where C > and c G (0, 1) depend (polynomially) only on B. 

Remarks. 1. For e — 0, Theorem 11.21 yields an exponential bound for the 
singularity probability: 

P (random subgaussian matrix A is singular) < c n . 



In the literature in geometric functional analysis, the subgaussian moment is often called 
the ?/>2-norm. 
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Thus Kahn-Komlos-Szemeredi's bound (11.61) holds for all subgaussian matrices. 
Moreover, while (JT75J) estimates the probability that a random matrix belongs 
to the set of singular matrices, Theorem 11.21 estimates the distance to that set. 

2. The bounds in Theorem 11.21 are precise. Edelman's bound (I1.3P shows 
that the term en~ x l 2 is optimal for the Gaussian matrix, while the term c n is 
optimal for a random sign-matrix. 

3. For simplicity, we state and prove all our results over the real field. 
However, our arguments easily generalize to the complex field; see e.g. [2TJ . 

4. A weaker result was recently proved by the first author [22] who showed 
that F(s n (A) < en~ 3//2 ) < Ce + Cn^ 1 / 2 . He later improved the term n~ v l 2 
to c n . Shortly after that, both authors of this paper independently discovered 
how to reduce the term n _3//2 to the sharp order n^ 1 ^ 2 . In December 2006, 
the second author found a new way to prove the sharp invertibility estimate 
by obtaining an essentially optimal result for the Littlewood-Offord problem 
as stated in Theorem 11.51 We thus decided to publish jointly, and some of the 
arguments were improved during the final stage of our work. 

5. Another weaker result was recently proved by Tao and Vu [31] for random 
sign matrices. They showed that for every A > there exists B > such that 
Sn(A) > n~ B holds with probability 1 — OA{n~ A ). 

1.2. The Littlewood-Offord problem. Our results on random matrices 
come as a consequence of a new and essentially sharp estimate in the Littlewood- 
Offord problem [2], [10J. A classical theme in Probability Theory is the study 
of the random sums 



where £i, • • • ,£ n are independent identically distributed random variables and 
a = (dx, . . . , a n ) is a vector of real coefficients. 

The large deviation theory demonstrates that 5* nicely concentrates around 
its mean. On the other hand, by the central limit theorem, one can not expect 
tighter concentration than that of the appropriately scaled Gaussian random 
variable. However, rigorous anti-concentration estimates are hard to prove 
(see [H]), especially for discrete random variables The Littlewood-Offord 
problem thus asks to estimate the small ball probability 



A small value of p £ (a) would mean that the random sums S are well spread. 

For the random Gaussian sums, i.e. for being standard normal random 
variables, the small ball probability for each e depends only on the Euclidean 
norm of the coefficient vector a and not on its direction, and one has p e (a) ~ 
e/\\a\\ 2 . 



n 



(1.8) 




fc=i 



p e (a) := sup F(\S — v\ < e). 
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For most other distributions, p s (a) depends on the direction of a, and deter- 
mining the asymptotics is hard. A remarkable and extensively studied case is 
for the random sign-sums £±6^, i.e. for symmetric ±1 random variables 
The small ball probability strongly depends on the direction of the coefficient 
vector: for example, Po(a) = 1/2 for a = (1, 1, 0, . . . , 0) while Po(a) ~ n~ l l 2 for 
a=(l,l,...,l). 

The coefficient vectors with few non-zero coordinates turn out to be the only 
obstacle for nontrivial estimates on the small ball probability. The classical 
result of Littlewood and Offord strengthened by Erdos [7] states that if all 
\cik\ > 1 then for the random sign-sums one has 

(1.9) Pi(a)<n-^. 

This is sharp for = 1: there are lots of cancelations in most of the sign- 
sums £ ±1. However, if | aj — | > 1 for k ^ j, then the small ball probability 
is even smaller: 

(1.10) pM <n-*' 2 . 

This was proved by Erdos and Moser [8] for po(a) and with an extra logn 
factor, which was removed by Sarkozi and Szemeredi [23J. Halasz [12J proved 
this estimate for pi(a) and generalized it to higher dimensions. Estimate fll.lOp 
is sharp for at = k: there are still many cancelations in most of the sign-sums 

£±*- 

Tao and Vu [21] recently proposed a method to reduce the small ball proba- 
bility to an arbitrary polynomial order. They suggested to look at the inverse 
problem and to study the following phenomenon: 

If the small ball probability po(a) is large then the coefficient 

vector a has a rich additive structure. 
Thus, the only reason for many cancelations in the sign-sums =t a fc is that 
most coefficients at are arithmetically well comparable. By removing this ob- 
stacle one can force the small ball probability down to an arbitrary polynomial 
order: 

Theorem 1.3 (Tao, Vu [HE])- Let ai,...,a n be integers, and let A > 1, e G 

(0, 1). Suppose for the random sign-sums one has 

p (a) > n~ A . 

Then all except OA,e{n £ ) coefficients at are contained in the Minkowski sum of 
0(A/e) arithmetic progressions of lengths n° A < e ^ . 

(Recall that the Minkowski sum of sets is defined as U + V = {u + v : uG 
U, v e V}.) 

In this paper we demonstrate that a similar, and even simpler, phenomenon 
holds for real rather than integer numbers a&, for the small ball probabilities 
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p £ (a) rather than the probability po(a) of exact values, and for general random 
sums fll.Sp rather than the random sign-sums. 

We thus provide an essentially sharp solution to the Littlewood-Offord prob- 
lem for coefficients of equal order of magnitude. We show that one can force 
the small ball probability p E (a) down to an arbitrary function of n, up to an 
exponentially small order, which is best possible. We prove that: 

The coefficients of a are essentially contained in one arithmetic 
progression of length ^p £ (a) _1 . 
By "essentially" we mean that for arbitrary a G (0, 1) and k > c(a) we can 
guarantee that all but k coefficients at are within ad from the elements of some 
arithmetic progression, where d is the gap between its elements. It is convenient 
to state this result in terms of the essential least common denominator of real 
numbers: 

Definition 1.4 (Essential LCD). Let a G (0,1) and k > 0. The essential 
least common denominator D(a) = D a>K (a) of a vector a 6 I" is defined as 
the infimum of t > such that all except k coordinates of the vector ta are of 
distance at most a from nonzero integers. 

For numbers = 0(1), the essential LCD has an obvious interpretation 
in terms of arithmetic progressions: all except K coefficients are within 
distance a/D(a) = 0(a) from the elements of an arithmetic progression of 
length 0(D(a)). 

Theorem 1.5 (Small Ball Probability). Let £i, . . . , £ n be independent identi- 
cally distributed centered random variables with variances at least 1 and third 
moments bounded by B. Let a = (ai, . . . ,a n ) be a vector of real coefficients 
such that, for some K 1: K 2 > one has 



where C, c > depend (polynomially) only on B,K\, K^. 

A more precise version of this result is Theorem 14.11 below. 

Remarks. 1. By the definition, one always has D aK (a) > I/-K2 (e.g. with 
a = 1/3 and k = n/A). Theorem 11.51 thus yields pi(a) < nr 1 ! 2 , which agrees 
with Littlewood-Offord and Erdos inequality (11.91) . 

2. Suppose the components of a are uniformly spread between two compara- 
ble values; say a = (n, n+l, n+2, . . . , 2n). Obviously, D a<K (a/n) ~ n (e.g. with 
a = 1/3 and n = n/A). Theorem 11.51 thus yields pi(a) = pi/ n (a/n) < n~ 3 / 2 . 
This agrees with Erdos- Moser inequality (11.101) . 



(1.11) K 1 <\a k \<K 2 for all k. 

Let a G (0, 1) and k G (0, n). Then for for every e > one has 
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3. By making coefficients of a more arithmetically incomparable, such as by 
considering polynomial progressions, one can force the small ball probability 
p e (a) down to an arbitrarily small value, up to an exponentially small order. 

One can restate Theorem 11.51 as an inverse Littlewood-Offord theorem: 

Corollary 1.6 (Inverse Littlewood-Offord Theorem). Let ai,...,a n be real 
numbers satisfying (II. lip and£i, . . . , £ n be random variables as in Theorem \1.5l 
Let A > 1/2, k G (0,n) and e > 0. Suppose for the random sums (11.81) one 
has 

p £ (a) > n . 

Then there exists an arithmetic progression of length L = 0(n A n~ 1 ^ 2 ) and 
with gap between its elements d < 1, and such that all except k coefficients au 
are within distance 0(A log(n)//t) 1//2 • d from the elements of the progression, 
provided that e < 1/L. 

By Remark 1 above, the assumption A > 1/2 is optimal. 

In contrast with Theorem 11.31 Corollary 11.61 guarantees an approximate, 
rather than exact, embedding of the coefficients into an arithmetic 

progression. On the other hand, Corollary 11.61 (a) applies for real rather 
integer coefficients; (b) embeds into one arithmetic progression rather than 
a Minkowski sum of several progressions; (c) provides a significantly sharper 
bound on the length of the progression; (b) characterizes general small ball 
probabilities p e (a) rather than the probability of exact values po(a); (d) holds 
for general sums of i.i.d. random variables rather than the random sign-sums. 

1.3. Outline of the argument. We develop a general approach to the in- 
vertibility of random matrices. Our main result, the Strong Invertibility The- 
orem 15.11 reduces estimating the smallest singular value of random matrices 
to estimating the largest singular value. Because the largest singular value 
is much more studied, this immediately implies both our invertibility results 
stated above, Theorems 11.11 and 11.21 

The general approach to invertibility is developed in two stages. In Section [3] 
we present a "soft" and rather short argument that leads to a weaker result. 
It yields the Fourth Moment Theorem 11.11 and also a weaker version of the 
Subgaussian Theorem 11.21 with Cn~ l l 2 instead of the exponential term c n in 

(H2D- 

Our soft argument does not use any new estimates of the small ball proba- 
bility. To bound || Ac|| 2 below for all vectors x in the unit sphere, we give two 
separate arguments for compressible vectors x, whose norm is concentrated in 
a small number of coordinates, and for incompressible vectors comprising the 
rest of the sphere. 

For a compressible vector, the main contribution in the quantity ||Ac||2 
comes from the few (say, n/10) columns of A corresponding to the biggest 
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coordinates of x. This allows us to replace A by its nxn/10 submatrix with 
the chosen columns. Such rectangular random matrices are known to have big 
smallest singular value (see e.g. p2]), which establishes a nice lower bound on 
||Ar|| 2 for all compressible vectors. 

For the incompressible vectors, we show the invertibility differently. Clearly, 
s n (A) is bounded above by the distance from its n-th row vector X n to the span 
H n of the others. We use a careful average union argument (Lemma 13. 5p to 
show a reverse inequality for A restricted to the set of incompressible vectors. 

Next, this distance can be bounded below as dist(X n ,H n ) > \(X*,X n )\, 
where X* is a unit normal of H n . Since X* and H n are independent, the inner 
product (X*, X n ) can be written as a sum of independent random variables of 
the form (jl.8p . This reduces the invertibility problem to the Littlewood-Offord 
problem. 

A useful small ball probability bound can be deduced from the central limit 
theorem, by approximating the random sum (jl.8p with a Gaussian random 
variable for which the small ball probability is easy to compute. With such 
bound, the argument above yields a weaker version of the invertibility estimate 
( TO) with Cn~ 1 / 2 instead of c n . 

This weaker estimate is a limitation of using the central limit theorem. To 
prove the Strong Invertibility Theorem 15.14 and thus deduce the Subgaussian 
Theorem 11.21 we will use the full strength of the Small Ball Probability The- 
orem 11.51 instead. This argument is presented in Section [5j 

Our proof of Theorem 11.51 starts with the method developed by Halasz [TTI 
IT2] . It allows us to bound the small ball probability p £ (a) by a quantity of 
ergodic nature - the measure of the recurrence set of a. It indicates how often 
a particle in R n moving in the direction a with unit speed gets close to the 
points of the integer lattice. If this happens often, then a density argument 
shows that the particle must get close to two distinct lattice points over a short 
period of time, say at times t\ and £2- It then follows that (£2 —t\)a is close to 
an integer, which implies that the essential LCD of a is small. This argument 
is given in Section HI 

Acknowledgement. The authors are grateful to the referee for the careful 
reading of the manuscript and valuable suggestions. 

2. Preliminaries 

In the sequel n denotes a sufficiently large integer, i.e. an integer bigger than 
a suitable absolute constant. The standard inner product on R™ is denoted 
by (x,y). The l v norm on R™ is defined as = (Y^k=i \ x k\ p ) 1 ^ p for < 
p < 00, and ||a:||oo = maxfc \xk\- The unit Euclidean ball and the sphere in R n 
are denoted by and S"" 1 respectively. For a subset a C {1, . . . ,n}, the 
orthogonal projection onto W 7 in R™ is denoted by P a . 
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The following observation will allow us to select a nice subset of the coeffi- 
cients a k when computing the small ball probability. 

Lemma 2.1 (Restriction). For any a £ MJ 1 , any a C {l,...,n} and any 

e > 0, we have 

Pe(a) < Pe(Paa). 

Proof. For fixed v £ K. and for the random sum (11. 8p . we write S — v = S a — v a , 
where S a := J2ke<r a A and v a := v— J2 k( z a c a k ^ k . We condition on a realization 
of (£k)k£a c , and denote by P CT the probability with respect to (£k)kecr- Then a 
realization of v a is fixed, so 

K(\S -v\<e)= F ff (\S a - v a \ <s)< p £ (P a a). 

Taking the expectation of both sides with respect to (£ k ) keaC completes the 
proof. □ 

The following tensorization lemma transfers one-dimensional small ball prob- 
ability estimates to the multidimensional case. It is a minor variant of Lemma 
4.4 of [22]. 

Lemma 2.2 (Tensorization). Let Ci, • • ■ , Cn be independent non-negative ran- 
dom variables, and let K,Eq>0. 

(1) Assume that for each k 

P(Cfc <e)<Ke for all e > e . 

Then 

n 

p ( E ^ < e2n ) ^ ( CK£ ^ n f° r aU £ ^ 
fc=i 

where C is an absolute constant. 

(2) Assume that there exist A > and // £ (0, 1) such that for each k 

^ < a) < n. 

Then there exist X 1 > and fix £ (0, 1) that depend on A and \x only and such 
that 

n 
k=l 

We give a proof of the first part for completeness. The second part is similar, 
cf. [TF] proof of Proposition 3.4. 
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Proof. Let e > Eq. By Chebychev's inequality, 

n \ n 1 " 

p(^C fc 2 <£ 2 n) =p(n--^C. 2 >0) <Eexp (n - -^C 2 ) 

fc=l fc=l k=l 

n 

(2-1) =e"n Eex P(-C^ 2 )- 

fc=i 

By the distribution integral formula, 

poo 

Eexp(-Cfc/e 2 ) = / F(exp(-(t/£ 2 ) > s) ds = / lue^ P(0k < eu) du. 
Jo Jo 

For k e (0, 1), we have f((k < £u) < P(Cfc < e) < -Ke- This and the assump- 
tion of the lemma yields 

/l /'OO 
2we- u2 Ke + y 2Me" u2 il"£M du < CKe. 

Putting this into (12.1 j) yields 

n 



fc=l 



This completes the proof. □ 

2.1. Largest singular value. We recall some known bounds on the largest 
singular value of random matrices under the fourth moment assumption and 
the subgaussian moment assumption. The following result is a partial case of 
a recent result of Latala. 

Theorem 2.3 (Largest singular value: fourth moment P3]). Let A be an n x n 

matrix whose entries are independent centered random variables with variances 
at least 1 and fourth moments bounded by B. Then 

Ep|| < Cm 1/2 

where C\ = CB l l 4 , and where C is an absolute constant. 

Under the stronger subgaussian moment assumption, a standard observation 
shows that \\A\\ ~ n 1 / 2 with exponentially large probability (see e.g. [I] and 
PU, Fact 2.4): 

Lemma 2.4 (Largest singular value: subgaussian). Let A be an n x n matrix 
whose entries are independent centered random variables with variances at least 
1 and subgaussian moments bounded by B. Then 

P(||A|| > Cm 1 ' 2 ) < 2e~ n , 

where C\ depends only on B. 
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2.2. Smallest singular value of rectangular matrices. Estimates on the 
smallest singular value are known for rectangular random matrices [T7] . 

Proposition 2.5 (Smallest singular value of rectangular matrices). Let G be 

an n x k matrix whose entries are independent centered random variables with 
variances at least 1 and fourth moments bounded by B. Let K > 1. Then 
there exist ci, c% > and So G (0, 1) that depend only on B and K such that if 
k < Son then 

(2.2) P( inf \\Gx\U < c x n 1/2 and \\G\\ < Kn 1/2 ) < e~ C2n . 

Under the stronger subgaussian assumption, the condition \\G\\ < Kn 1 ^ 2 
can clearly be removed from (12.2ft by Lemma 12. 41 This is not so under the 
fourth moment assumption. So here and later in the paper, this condition will 
often appear in order to deduce the Fourth Moment Theorem 11.11 The reader 
interested only in the Subgaussian Theorem 11.21 can disregard this condition. 

A result stronger than Proposition I2.5[ for the aspect ratio So arbitrarily 
close to 1, follows by modifying the argument of [TF]. For completeness, we 
shall prove Proposition 12.51 We start with the most general (but weakest 
possible) estimate on the small ball probability. 

Lemma 2.6. Let £i, be independent centered random variables with 

variances at least 1 and fourth moments bounded by B. Then there exists 
/i G (0, 1) depending only on B, such that for every coefficient vector a = 
(at, . . . , a n ) G S 1 ™ -1 the random sum S = Ylk=i a k£,k satisfies 

F(\S\ < 1/2) < fi. 

Proof. Let E\,...,e n be independent symmetric ±1 random variables, which 
are independent of £i, . . . , £ n . By the standard symmetrization inequality (see 
[To] Lemma 6.3), 

n 4 

ES 4 < 16E(53e fc &a fc ) • 

fc=i 

We first condition on £i,...,£ n and take the expectation with respect to 
£i, . . . ,£ n - Khinchine's inequality (see e.g. [16] Lemma 4.1) and our assump- 
tions on £k then yield 

fc=l k,j=l 
n n 2 

< c j2 m) 1/2 m*y /2 aia 2 < CB{^l) = CB. 

k,j=l k=l 
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The Paley-Zygmund inequality (see e.g. [17], Lemma 3.5) implies that for any 
A > 

n/lnl x , (ES 2 -A 2 ) 2 (1-A 2 ) 2 
n\s\ > A) > ^—L > L—L. 

To finish the proof, set A = 1/2. □ 

Combining Lemma 12.61 with the tensorization Lemma 12.21 we obtain the 
following invertibility estimate for a fixed vector. 

Corollary 2.7. Let G be a matrix as in Proposition \2.5[ Then there exist 
constants r),v G (0, 1) depending only on B, such that for every x G S k ^ 1 

P(||Gz|| 2 < 7]n 1/2 ) < v* 

Proof of Proposition \2.5\ Let e > to be chosen later. There exists an e-net 
J\f in S ,/c_1 (in the Euclidean norm) of cardinality \Af\ < (3/e) fc (see e.g. [19]). 
Let r\ and v be the numbers in Corollary 12.71 Then by the union bound, 

(2.3) P(3xGAT: \\Gx\\ 2 <r]n 1/2 ) < (3/e) k -u n . 

Let V be the event that \\G\\ < Kn 1 / 2 and ||Gfy|| 2 < \rirt}! 2 for some point 
y G S^ 1 . Assume that V occurs, and choose a point x G H such that 
— x|| 2 < e. Then 

\\Gx\\ 2 < \\Gy\\ 2 + ||G|| • ||x - y\\ 2 < ^r/n 1/2 + Kn 1 ' 2 ■ e = r]n 1/2 , 
if we set e = r]/2K. Hence, by (12.31) . 

F(V) < (v(3/e) k/n ) n < e- C2n , 

if we assume that k/n < 5q for an appropriately chosen 5q < 1. This completes 
the proof. □ 

2.3. The small ball probability via the central limit theorem. The 

central limit theorem can be used to estimate the small ball probability, as 
observed in [17]. Specifically, one can use the Berry-Esseen version of the 
central limit theorem (see [27], Section 2.1): 

Theorem 2.8 (Berry-Esseen CLT). Let d, . . . , ( n be independent centered ran- 
dom variables with finite third moments, and let a 2 := ^)? =1 E|£fc| 2 . Consider 
a standard normal random variable g. Then for every t > 0: 

1 n n 

(2.4) | P (~Z)Cfc<f) -H9<t)\ <Ca- 3 ^E|C fc | 3 , 

k=l k=l 

where C is an absolute constant. 

The following corollary is essentially given in |17| . We shall include a proof 
for the reader's convenience. 
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Corollary 2.9 (Small ball probability via CLT). Let £1, ■ ■ ■ ,£ n be independent 
centered random variables with variances at least 1 and third moments bounded 
by B. Then for every a G W 1 and every e > 0, one has 

V 7r ||a|| 2 V ||a||2/ 

where C\ is an absolute constant. 

Proof. We shall use Theorem [231 for (k = akik- There, a > ||a|| 2 and Y^l=i ^ICfc| 3 — 

1 1 3 r 

B \\a\\ 3 . Thus for every u G K we have 




Since the density of the standard normal random variable g is uniformly 
bounded by 1/ V2n, we have 

V(\g-u <t)< — = J-t. 

V27T V 7T 

With u = and t = the left hand side of (Q equals F(\S - v\ < e), 
which completes the proof with C\ = 2C. □ 

As an immediate corollary, we get: 

Corollary 2.10 (Small ball probability for big e). Let £i, . . . , £ n be independent 
centered random variables with variances at least 1 and third moments bounded 
by B. Assume that a coefficient vector a satisfies (11. 111) . Then for every e > 
one has 

p £ (a)<^ (e/K. + BiK./K^ 3 ), 
where Ci is an absolute constant. 

3. INVERTIBILITY OF RANDOM MATRICES: SOFT APPROACH 

In this section, we develop a soft approach to the invertibility of random 
matrices. Instead of using the new estimates on the small ball probability, 
we will rely on the central limit theorem (Corollary 12. 10p . This approach will 
yield a weaker bound, with polynomial rather than exponential term for the 
singularity probability. In Section [5] we shall improve upon the weak point of 
this argument, so the Small Ball Probability Theorem II .51 will be used instead. 

Theorem 3.1 (Weak invertibility). Let A be an n x n matrix whose entries 
are independent random variables with variances at least 1 and fourth moments 
bounded by B. Let K > 1. Then for every e > one has 

(3.1) f(s n (A) < en~ l/2 ) <Ce + Cn' 112 + P(||A|| > Kn 1/2 ), 
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where C depends (polynomially) only on B and K . 

To make this bound useful, we recall that the last term in (13 .ip can be 
bounded using Theorem 12.31 under the fourth moment assumption and by 
Lemma 12.41 under the subgaussian assumption. In particular, this proves 
Fourth Moment Theorem ll.il 

Proof of the Fourth Moment Theorem \l.l\ Let 5 > 0. By Theorem 12.31 and 
using Chebychev's inequality, we have 



Then setting K = 3Ci/5, e = 5/3C and n = (3(7/(5) 2 , we make each of the 
three terms in the right hand side of (13. ip bounded by 8/3. This completes 



Remark. Theorem 13. II in combination with Lemma I2~l4l yields a weaker version 
of the Subgaussian Theorem 11.21 with Cn^ 1 ! 2 instead of c n . 

3.1. Decomposition of the sphere. To prove Theorem 13. 1\ we shall parti- 
tion the unit sphere S"™ -1 into the two sets of compressible and incompressible 
vectors, and will show the invertibility of A on each set separately. 

Definition 3.2 (Compressible and incompressible vectors). Let 5, p £ (0,1). 
A vector x £ M n is called sparse if |supp(x)| < 8n. A vector x £ S^ 1 is called 
compressible if x is within Euclidean distance p from the set of all sparse vec- 
tors. A vector x £ S*™ -1 is called incompressible if it is not compressible. 
The sets of sparse, compressible and incompressible vectors will be denoted by 
Sparse = Sparse(S), Comp = Comp(5,p) and Incomp = Incomp(5, p) respec- 
tively. 

Remarks. 1. Here we borrow the terminology from the signal processing 
and the sparse approximation theory. Efficient compression of many real-life 
signals, such as images and sound, relies on the assumption that their coeffi- 
cients (Fourier, wavelet, frame etc.) decay in a fast way. Essential information 
about the signal is thus contained in few most significant coefficients, which 
can be stored in small space (see [SI E]). Such coefficient vector is close to a 
sparse vector, and is thus compressible in the sense of our definition. 

2. Sets similar to those of compressible and incompressible vectors were 
previously used for the invertibility problem in [17] and [22 j . 

3. In our argument, the parameters 5, p will be chosen as small constants 
that depend only on B and K. 



P Mil > 




the proof. 



□ 
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Using the decomposition of the sphere S"™ -1 = Comp U Incomp, we break 
the invertibility problem into two subproblems, for compressible and incom- 
pressible vectors: 

(3.2) ¥(s n (A) < en' 1 ' 2 and ||A|| < Kn 1/2 ) 

< P( inf ||Ar|| 2 < en- 1/2 and \\A\\ < Kn 1 ' 2 ) 

x£Comp(8,p) 

+ P( inf \\Ax\\ 2 < en- 1/2 and ||A|| < Kn 1 ' 2 ). 

x&Incomp(8,p) 

The compressible vectors are close to a coordinate subspace of a small di- 
mension 5n. The restriction of our random matrix A onto such a subspace is a 
random rectangular n x 5n matrix. Such matrices are well invertible with ex- 
ponentially high probability (see Proposition 12.51) . By taking the union bound 
over all coordinate subspaces, we will deduce the invertibility of the random 
matrix on the set of compressible vectors. 

Showing the invertibility on the set of incompressible vectors is generally 
harder, for this set is bigger in some sense. By a careful average union argu- 
ment, we shall reduce the problem to a small ball probability estimate. 

3.2. Invertibility for the compressible vectors. On the set of compress- 
ible vectors, a much stronger invertibility holds than we need in (13.21) : 

Lemma 3.3 (Invertibility for compressible vectors). Let A be a random matrix 
as in Theorem \3.1[ and let K > 1. Then there exist 5, p, C3, C4 > that depend 
only on B and K , and such that 

P( inf ||Ae|| 2 < c 3 n 1/2 and \\A\\ < Kn 1/2 ) < e~ Cin . 

xEComp(5,p) 

Remark. The bound in Lemma 13.31 is much stronger than we need in (13.21) . 
Indeed, by choosing the constant C in Theorem 13.11 large enough, we can 
assume that n > I/C3 and e < 1. Then the value c^n 1 ^ 2 in Lemma [3731 is bigger 
than en~ x l 2 in (J37J). 

Proof. We first prove a similar invertibility estimate for the sparse vectors. To 
this end, we can assume that So < 1/2 in Proposition 12.51 We use this result 
with k = 5n and take the union bound over all \8n\ -element subsets o of 
{l,...,n}: 

(3.3) P( inf ||Az?|| 2 < cin 1/2 and \\A\\ < Kn 1/2 ) 

xdSparse{5), ||z||2=l 

= P(3<7, |ct| = \8n\ : inf \\Ax\U < c x n xl2 and IUII < Kn 1/2 ) 

xeM' T , ||x||a=l 

< ( rx n j e-° 2n < exp(4e51og(e/5)n - c 2 n) < e - c * n/2 
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with an appropriate choice of 5 < Sq, which depends only on c 2 (which in turn 
depends only on B and K). 

Now we deduce the invertibility estimate for the compressible vectors. Let 
c 3 > and p G (0, 1/2) to be chosen later. We need to bound the event V that 
||Ar|| 2 < c^n 1 ! 2 for some vector x G Comp(5, p) and ||A|| < Kn 1 ^ 2 . Assume 
V occurs. Every such vector x can be written as a sum x = y + z, where 
y G Sparse(5) and ||z|| 2 < p. Thus \\yW2 > 1 — P > 1/2, and 

\\Ay\\ 2 < \\Ax\\ 2 + || A|| ||z|| 2 < c 3 n 1/2 + pKn 1 ' 2 . 

We choose c 3 := ci/4 and p := C\/^K so that ||Ay|| 2 < ^Cira 1 / 2 . Since ||y|| 2 > 
1/2, we have found a unit vector u G Sparse(S) such that ||Au|| 2 < cin 1 ^ 2 
(choose u = 2// 1 1 2/ 1 1 2) - This shows that the event V implies the event in (13.31) . 
so we have W(V) < e~ C2n / 2 . This completes the proof. □ 

3.3. Invertibility for the incompressible vectors via distance. For the 

incompressible vectors, we shall reduce the invertibility problem to a lower 
bound on the distance between a random vector and a random hyperplane. 

We first show that incompressible vectors are well spread in the sense that 
they have many coordinates of the order n^ 1 ^ 2 . 

Lemma 3.4 (Incompressible vectors are spread). Let x G Incomp(5, p) . Then 
there exists a set a C {1, . . . , n} of cardinality \cr\ > \p 2 b~n and such that 

' < \%k\ ^ for all k G a . 



2n y5n 
Proof. Consider the subsets of {1, . . . , n} defined as 

a x := {k : \x k \ < -7=}, cr 2 := {k : |x fe | > — ^=}, 
yon V 2n 



and put <t '.— <j\ n cr 2 . 

By Chebychev's inequality, \a\\ < 8n. Then y := P a <c G Sparse(5n), so the 
incompressibility of x implies that ||P CT1 a;|| 2 = \\x — y|| 2 > p. By the definition 
of <t 2 , we have ||-Ptr§a;||| < n ■ ^ = p 2 /2. Hence 

(3-4) ||P CT x|| 2 > ||P CTl x|| 2 - ||P CTi a;|| 2 > p 2 /2. 

On the other hand, by the definition of o~\ ~D a, 

(3-5) ||P CT x|| 2 < \\Pvx\ll ■ \a\ <^--\(t\. 

on 

It follows from Q and < B) that > \p 2 $n. □ 

Lemma 3.5 (Invertibility via distance). Let A be any random matrix. Let 
Xi, . . . ,X n denote the column vectors of A, and let H k denote the span of all 
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column vectors except the k-th. Then for every 5, p G (0, 1) and every e > 0, 
one has 

1 - 

(3.6) P( inf \\Ax\\ 2 < epn' 1 ' 2 ) < — V P(dist(X fc , H k ) < e). 

fc=l 

Remark. The main point of this bound is the average, rather than the max- 
imum, of the distances in the right hand side of (13.61) . This will allow us to 
avoid estimating the union of n events and thus bypass a loss of the n factor 
in the invertibility theorem. 

Proof. Let x G Incomp(5, p). Writing Ax = ^Jl = iX k X kl we have 
H^Mh > raa.x dist(Ax,H k ) 

k=l,...,n 

(3.7) = max dist(x k X k , H k ) = max \x k \ dist(X&, H}~). 

k=l,...,n k=l,...,n 



Denote 
Then 



Pk :=¥(dist(X k ,H k )<e). 



E\{k: dist(X k ,H k ) <s}\ =J2 



Pk- 



k=l 

Denote by U the event that the set o~\ := {k : dist(X k , H k ) > e} contains 
more than (1 — 5)n elements. Then by Chebychev's inequality, 

n 
k=l 

On the other hand, for every incompressible vector x, the set o~ 2 (x) := {k : 
\x k \ > pn~ l l 2 } contains at least 5n elements. (Otherwise, since ||-Pcr 2 (x) c ^||2 < 
p, we would have \\x — y\\ 2 < p for the sparse vector y := P a2 ^x, which would 
contradict the incompressibility of x). 

Assume that the event U occurs. Fix any incompressible vector x. Then 
+ I^O^)! > (1 — $) n + Sn > n, so the sets U\ and o~ 2 {x) have nonempty 
intersection. Let k G (J\ PI o 2 {x). Then by (j3.7p and by the definitions of the 
sets o"i and a 2 (x), we have 

||Ac||2 > \x k \dist(X k , H k ) > pn^ 1 / 2 ■ e. 
Summarizing, we have shown that 



P( inf \\Ax\\ 2 < epn- 1 ' 2 ) < F(U C ) <^V ft . 
This completes the proof. □ 



18 



MARK RUDELSON AND ROMAN VERSHYNIN 



3.4. Distance via the small ball probability. Lemma [3.51 reduces the in- 
vertibility problem to a lower bound on the distance between a random vector 
and a random hyperplane. Now we reduce bounding the distance to a small 
ball probability estimate. 

Let Xi, . . . , X n be the column vectors of A. These are independent random 
vectors in M™. Consider the subspace H n = span(Xi, . . . , X n _i). Our goal is to 
bound the distance between the random vector X n and the random subspace 
H n . 

To this end, let X* be any unit vector orthogonal to X 1; . . . , X n _i. We call 
it a random normal. We can choose X* so that it is a random vector that 
depends only on Xi, . . . , X n _i and is independent of X n . 

We clearly have 

(3.8) dist(X n ,#„) > \(X*,X n )\. 

Since the vectors X* =: (ai, . . . , a n ) and X n =: (£i, . . . , £ n ) are independent, 
we should be able to use the small ball probability estimates, such as Corol- 
lary EHU1 to deduce a lower bound on the magnitude of 

n 

(X*,X n ) = 2^ a k£k- 
k=l 

To this end, we first need to check that the coefficients of the vector X* are 
well spread. 

Lemma 3.6 (Random normal is incompressible). Let 5,p,C4 > be as in 
Lemma Iff. 31 Then 

P(X* G Comp(5,p) and \\A\\ < Kn 1/2 ) < e~ C4n . 

Proof. Let A' be the (n-l)xn random matrix with rows Xi, . . . , X n _i, i.e. 
the submatrix of A T obtained by removing the last row. By the definition of 
the random normal, 

(3.9) A'X* = 0. 

Therefore, if X* 6 Comp(5,p) then in.f X £Com P {5,p) ll^'^lh = 0. By replacing n 
with 7i — l, one can easilty check that the proof Lemma 13.31 remains valid for 
A' as well as for A; note also that \\A'\\ < \\A\\. This completes the proof. □ 

Now we recall our small ball probability estimate, Corollary I2.10| in a form 
useful for the incompressible vectors: 

Lemma 3.7 (Small ball probability for incompressible vectors). Let £i, . . . , £ n 

be random variables as in Corollary \2.1(K Let 5, p G (0,1), and consider a 
coefficient vector a G Incomp(5, p). Then for every e > one has 

p s {a)<C,{e + Bn- l l 2 ), 
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where C$ depends (polynomially) only on 5 and p. 

Proof. Let a denote the set of the spread coefficients of a constructed in 
Lemma 13.41 Then |er| > ^p 2 Sn, and the vector b := n l l 2 P a a satisfies K\ < 

\bk\ < K 2 for all k 6 a, where Ki = p/y/2 and K 2 = l/VS- By Restriction 
Lemma [2. II and Corollary 12.1 0} we have 

Pe(a) = Vn^M l2 a) < Pn i/2 E (b) <C b {e + Bn x l 2 ). 

This completes the proof. □ 

Lemmae 13.71 and 13.61 imply the desired distance bound: 

Lemma 3.8 (Weak Distance Bound). Let A be a random matrix as in Theo- 
rem \3.1\ Let X\, . . . ,X n denote its column vectors, and consider the subspace 
H n = span(Xi, . . . , X n -i). Let K > 1. Then for every e > 0, one has 

P(dist(X n , H n ) < e and \\A\\ < Kn 1/2 ) < C^E + n- 1 ' 2 ), 

where Cq depends only on B and K . 

Remark. In Theorem 15.21 below, we shall improve this distance bound by re- 
ducing the polynomial term n^ 1 ^ 2 by the exponential term e~ cn . 

Proof. We condition upon a realization of the random vectors Xi, . . . 
This fixes realizations of the subspace H n and the random normal X*. Recall 
that X n is independent of X*. We denote the probability with respect to X n 
by P n , and the expectation with respect to Xi, . . . , X n _i by E i) ... jn _i. Then 

(3.10) f(\{X*,X n )\ < e and \\A\\ < Kn 1/2 ) 

< E li ..., n _iP n (|(X*, J X n >| < e and X* e Incomp(6,p)) 

+ P(X* G Comp(5,p) and \\A\\ < Kn 1/2 ). 

Fix 5, p > so that the conclusion of Lemma [3761 holds. This bounds the last 
term in the right hand side of (13.101) by e~ C4n . Furthermore, by Lemma 13.71 
for any fixed realization of X 1; . . . , X n such that X* 6 Incomp(5, p) we have 

F n (\(X*,X n )\<e) <C' 5 (e + n-V 2 ), 

where C 5 depends only on B and K. It follows that 

¥{\{X\X n )\ < £ and \\A\\ < Kn 1/2 ) < C 5 (e + n~ l/2 ) + e" C4n . 

By ( 13.81) . the proof is complete. □ 

Combining Lemma 13.51 and Lemma 13. 8[ we have shown the invertibility of 
a random matrix on the set of incompressible vectors: 
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Lemma 3.9 (Invertibility for incompressible vectors). Let A be a random 
matrix as in Theorem \3.1\ Let K > 1 and 5, p G (0, 1). Then for every e > 0, 
one has 

P( inf \\Ax\\ 2 < epn~ l/2 ) < ^-(s + n~ 1/2 ) + P(||A|| > Kn 1/2 ), 

x£lncomp(S,p) 

where C-j depends only on B and K . 

3.5. Invertibility on the whole sphere. The Weak Invertibility Theorem 13. II 
now follows from the decomposition of the sphere (I3.2p into compressible and 
incompressible vectors, and from the invertibility on each of the two parts es- 
tablished in Lemma 13.31 (see the remark below it) and Lemma 13.91 (used for 
5, p as in Lemma 13.31 and for e/p rather than e). □ 

4. Small ball probability 

In this section, we prove the following more precise version of Theorem II .51 

Theorem 4.1 (Small Ball Probability). Let £ be a centered random variable 
with variance at least 1 and with the third moment bounded by B. Consider 
independent copies £i, . . . , £ n of £. Let a = (ax, ■ ■ • , a n ) be a coefficient vector 
and let K > 1 be such that 

(4.1) l<\a k \<K for all k. 

Let < a < 1/6K and < k < n. Then for every e > one has 

, s ^ CBK 3 ( 1 \ ^ / ca 2 
PeW < — ^(e + — — 1 +Cexp - — 

where C, c > are absolute constants. 



Remark. 1. This result clearly implies Theorem II .51 (Indeed, in Theorem 11.51 
one can assume that Kx — 1 by rescaling the coefficients a k , and that a < 
I/6K2 by considering a/6K 2 instead of a.) 

2. Since the definition of p e (a) includes shifts, Theorem 14.11 holds also for 
the shifted random variables = £j + tj for any real numbers tx, ■ ■ ■ , t n . 

The approach based on the central limit theorem establishes Theorem 14.11 
for the values of e of constant order and above. Indeed, for e > e > 0, 
Corollary 12.101 yields 

. x C'BK 3 

Pe{a) < 1=- £ 

>n 



where C' 2 depends only on e . 

For e below the constant order, this bound can not hold without any ad- 
ditional information about the coefficient vector a. Indeed, if all a k = 1 then 
random sign-sums satisfy po(a) > P(5 = 0) ~ n^ 1 ^ 2 . 
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We thus need to develop a tool sharper that the central limit theorem to 
handle smaller e. Our new method uses the approach of Halasz [TTj [T2"]. which 
was also used in 



4.1. Initial reductions, symmetrization, truncation. Throughout the 
proof, absolute constants will be denoted by C, c, ci, . . . The particular value 
of each constant can be different in different instances. 

As explained above, we can assume in the sequel that e is below a constant, 
such as 

(4.2) e < tt/4. 

We can also assume that k < n/2 and that > 1 by replacing, if necessary, 
6 by 

We shall symmetrize the random variables and remove any small values 
they can possibly take. For many random variables, such as random ±1, this 
step is not needed. 

Let £' be an independent copy of £ and define the random variable C := 
le-C'l- Then 

EC 2 = 2E|£| 2 > 2 and EC 3 < 8E|£| 3 < 8B. 
The Paley-Zygmund inequality (see e.g. [17], Lemma 3.5) implies that 

(EC 2 - D 3 1 

Denote by C the random variable C conditioned on C > 1. Formally, C is a 
random variable such that for every measurable function / one has 

E/(0 = p^ T yE/(C)l {c>1} . 

It then follows by (14. 3ft that for every measurable non-negative function /, one 
has 

(4-4) E/(C)>/?E/(C). 

4.2. Small ball probability via characteristic functions. An inequality 
of Esseen ([9], see also |12j). bounds the small ball probability of a random 
variable S by the L\ norm of its characteristic function 

(j)(t) = cj) S (t) = Eexp(iSt). 

Lemma 4.2 (Esseen's Inequality). For every random variable S and for every 
e > 0, one has 

rir/2 

supF(\S-v\ < e) < C / \<f>{t/e)\ dt, 



where C is an absolute constant. 
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We want to use Esseen's Inequality for the random sum S = Ylk=i a k£k- 
The characteristic function of a k ^k is 

(j) k {t) := Eexp(ia k £, k t) = Eexp(za fc £t), 

so the characteristic function of 5* is then 



k=l 



To estimate the integral in Esseen's Lemma [4.21 we first observe that 



b k (t)\ 2 =Ecos(a fc Ct). 



Using the inequality \x\ < exp(— 1(1 — x 2 )) valid for all x, we then obtain 
|0(*)|<nexp(--(l-|0,(t)| 2 )) 

k=l 

= exp ( - E J2 j (! - cos («fca))) = exp ( - E/(C*)) , 
k=i 

where 

n 1 
f(t) :=^sin 2 (-a fc t). 

fc=l 

Hence by (14 AL we have 

<exp(-/3E/(Ct)). 

Then by Esseen's Lemma 14.21 and using Jensen's inequality, we estimate the 
small ball probability as 

/ tt/2 i-tt/2 
\<p{t/e)\ dt<C exp (-/3 Ef{(t/e)) dt 
-tt/2 J-tt/2 

<CE [ ' exp(-/3f((t/e)) dt 

J -tt/2 

(4.5) < Csup / exp ( - f3f(zt/e)) dt. 

z>\ J-tt/2 

Fix z > 1. First we estimate the maximum 

■n 

M := max f(ztle) = max > sin 2 (at.2:£/2£). 

|t|< ff /2 M 7 ^ |t|<V2f^ V 7 ^ 

fc=l 

Lemma 4.3. We /iai>e 

n 

— < M < n. 
4 ~ ~~ 
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Proof. The upper bound is trivial. For the lower bound, we estimate the 
maximum by the average: 

M > i r l2 f(ztM it = i ± ( , _ ^iim. 

~ xJ-,/2 2^ V na t z/2e J 

By our assumptions, > 1, z > 1 and e < 7r/4. Hence irakz/2e > 2, so 

n „ / sin t \ n 
M > — inf ( 1 > -. 

This completes the proof. □ 

Now we consider the level sets of /, defined for m, r > as 

T(m,r) := {t : \t\ < r, f(zt/e) < m}. 

By a crucial lemma of Halasz, the Lebesgue measure of the level sets \T(m, r)| 
behaves in a regular way QI2j, see [22], Lemma 3.2): 

Lemma 4.4 (Regularity). Let I G N be such that l 2 m < M. Then 

\T(m^)\<j.\T(l 2 m,Tr)\. 
Hence, for every 77 e (0, 1) such that m < 77M, one has: 



(4-6) \T(m,^)\<4J^.\T( V M,n)\. 

(Apply Lemma H31 with Z = [J^\)- 

Now we can estimate the integral in (14. 5 p by the integral distribution for- 
mula. Using (14. 6 p for small m and the trivial bound \T(m, 7r/2)| < 7r for large 

m, we get 

/■tt/2 

Pe( a ) < C SU P / ex P ( — (3f( z t/ e )) dt 

2 >1 J-n/2 
poo 

<C \T(m,-)\ (3e- pm dm 
Jo 2 

<C 4j— ■ \T(riM^)\ (3e-^ m dm + C / tt /^e^" 1 dm 

Jo V ^ M ir,Af 

(4.7) < ^ . \T(r}n,7r)\ + Cne~ c ^ n / B \ 



In the last line, we used Lemma 14.31 and the definition (14.31) of f3. 
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4.3. Recurrence set. We shall now bound the measure of the level set \T(rjn, tt) 
by a quantity of ergodic nature, the density of the recurrence set of a. 
Consider any t G T(r]n, tt) and set y := z/2e. Then y > l/2e, and 

n 

(4.8) f{zt/e) = s™ 2 {a k yt) < V n. 

k=l 

Let us fix 

(4.9) , : — . 

Then at least n — k terms in the sum in (14. 8B satisfy 

■ it . rin a 2 1 
sin (a k yt) < — = — < 



K 



4 144' 



which implies for those terms that dist(akyt, 7rZ) < a. Thus yt/n belongs to 
the recurrence set of a, which we define as follows: 

Definition 4.5 (Recurrence set). Let a G (0, 1) and k > 0. The recurrence 
set 1(a) = I ajK (a) of a vector a G M n is defined as the set of all t G K such that 
all except k coordinates of the vector ta are of distance at most a from Z. 

Regarding i as time, we can think of the recurrence set as the moments 
when most of the particles moving along the unit torus with speeds a 1; . . . , a n 
return close to their initial positions. 

Our argument thus shows that T(r]n,n) C -I atK (a). Thus 
|T(77n,7r)| < \-I atK (a) n [— k,it]\ = - ■ \I a , K (a) n 



The quantity 



dens(/,y) := • |/n [-j/,y]| 



can be interpreted as the density of the set /. We have thus shown that 

|T(r/n, 7r)| < 2ndens(I a ^(a),y). 
Using this bound and our choice (14.91) of rj in (14 .?p . we conclude that: 

(4.10) p E (o) < — ^ • sup dens(I a , K (a),y) + 0^-^^^ . 

a V K y>l/2e 
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4.4. Density of the recurrence set. It remains to bound the density of the 
recurrence set 1(a) by the reciprocal of the essential LCD D(a). We will derive 
this from the following structural lemma, which shows that: (1) the recurrence 
set has lots of gaps; (2) each gap bounds below the essential LCD of a. 
For t G M, by [t] we denote an integer nearest to t. 

Lemma 4.6 (Gaps in the recurrence set). Under the assumptions of Theo- 
rem \4-l\ let t G I a ^(a). Then: 

(1) t + 3a &I QiK (a). 

(2) Let t 1 G be such that ti > t + 3a. Then t\ — 1 > D 2a ,2K( a )- 

Since D 2a ^ 2K (a) > (1 — 2a)/ K > 4a, this lemma implies that the recurrence 
set / has gaps of size at least D 2a2K (a) — 4a. 

Proof. Part 1. Since t G / a , K (a), there exists a set cr <~ {l) - j n } °f 
cardinality |<7o| > n — k and such that for pk : = [ioOfc] we have: 

(4.11) |^o a fc ~ Pk\ < ol for all k G (To- 
Let t := t + 3a. Recall that 1 < a k < K for all k G {1, . . . , n}. By fliAil . 

we have for all k G cr : 

(4.12) ta fc = t a/c + 3a • a fc > p fc - a + 3a > p fc + a; 

tctfc < P/t + « + 3a • afc < pfc + a + 1/2 < p fc + 1 — a. 

In the last inequality, we used the assumption a < 1/6K < 1/6. It follows 
that dist(tafc, Z) > a for all k G o-q. Thus t G" I a , K (a). Part 1 is proved. 

Part Since t x G / a , K (a), there exists a set o~x C {1, . . . , n} of cardinality 
> n — k, and such that for '■— \t\Ok\ we have: 

(4.13) \t\Ok — < a for all k G 0i. 

Set a := (7 n oi. Then [cr[ > n - 2k. Moreover, ffl~TB and fflTTSj) yield: 

| (ti - t W - {qk ~Pk)\ < 2a for all k G a. 
Since ti > t, (14.121) implies that 

(4.14) t\Ok > tau > pk + a for all k G a. 

Hence, by ( I4.13P and f)4.14p . q k — Pk > for all k G a. By the definition of the 
essential LCD, this means that 

ti - to > D 2a ^(a). 

This completes the proof. □ 

We can use Lemma 14.61 to bound the density of the recurrence set via the 
reciprocal of the essential LCD. 
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Lemma 4.7 (Recurrence set via essential LCD). Under the assumptions of 
Theorem \4-l\ we have for every y > 0: 

(4.15) dens(I a>K (a),y) < 3a(-^ + 2 ). 

Remark. The contribution of the first term in (14.151) comes from the 0(a)- 
neighborhood of zero, which is contained in the recurrence set. This is the 
initial time when all of the moving particles are still close to 0. 

Proof. Denote / := I atK (a) fl [— y,y]. This set is closed and nonempty (it 
contains 0). Set to := min{t : t G /}. If / C [to,t + 3a], then 

(4.16) dens(J,y) = H<^ ) 

% 2y 

which completes the proof in this case. 

Assume then that / $2 [to, to + 3a]. Then we can define inductively the 
maximal sequence of points t±, t 2 , • • • , ^r, £ I by 

ti : = min{t : t 6 /; t > U-\ + 3a}. 

Note that by Lemma 14.61 + 3a I. Thus the strict inequality in the 
definition of t\ can be replaced by the non-strict inequality, so the minimum 
makes sense. 

Part 1 of Lemma 14.61 yields 

L 

ic(jfo,^ + 3a), 
1=0 

while part 2 implies 



t L ~ t > £}(t, - tz-i) > L ■ D 2a> 



1=1 



On the other hand, since t ,t L e I C [—y,y], we have t L — t < 2y. We 
conclude that 

dens (J, y) < llfcoMi + 3a)| < + *) ' 3a < 6a 



tL — to L ■ D 2a<2K (a) D 2a 2K (a) 

This completes the proof. □ 

By f )4.10p and Lemma [4.71 we conclude that 

Ma) < + lr L n ) + o™— V* 

V D 2a>2K (a)J 

for all e < 7r/4 (which was our assumption (14.21) .) 
This completes the proof of Theorem 14.11 
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4.5. Small ball probability for general coefficients. In view of the ap- 
plications, we will state Theorem 11.51 for a general coefficient vector a, not 
necessarily with well comparable coefficients as in (11. lip . This is easy to do 
by restricting a onto its spread part, which we define as follows: 

Definition 4.8 (Spread part). Let < K\ < K 2 be fixed. For a vector x G M n , 
we consider the subset a(x) C {1, . . . , n} defined as 

k G a(x) if K x < \n 1/2 x k \ < K 2 , 
and, if a(x) ^ 0, we define the spread part of x as 

X '.= {n I Xk)k£a(x)- 

If <t(x) = 0, the spread part of x is not defined. 

As an immediate consequence of Restriction Lemma 12.11 and Theorem 11.51 
we obtain: 

Corollary 4.9 (Small ball probability for general vectors). Let £i, . . . ,£ n be 

random variables as in Theorem \1.5[ Let a G M. n be a vector of real coefficients 
whose spread part a is well defined (for some fixed truncation levels Ki, K 2 > 
0/ Let a G (0, 1) and j3 G (0, 1/2). Then for for every e > one has 

Pe(a) <-^(e+ rj } pr ) + Ce^ n } 
y/j3\ y/nDu^a)/ 

where C,c> depend (polynomially) only on B,K\, K 2 . 

Remark. As a convention throughout the paper, we set D ajK (a) = if a is not 
defined. 

Remark. A small ball probability bound similar to Theorem 14. II can be proved 
with a weaker assumption on the coefficient vector. Namely, (14. ip can be 
replaced by 

IMIi > n i 1 1 a- 1 1 2 < K\fn. 
5. Invertibility of random matrices via small ball probability 

We return here to the invertibility problem for random matrices that we be- 
gan to study in Section [31 and we improve the Weak Invertibility Theorem 13.11 
by reducing the polynomial term n 1 ^ 2 to an exponentially small order c n . 

Theorem 5.1 (Strong invertibility). Let £ n be independent centered 

random variables with variances at least 1 and fourth moments at most B. Let 
A be an n x n matrix whose rows are independent copies of the random vector 
• • ■ >£n)- Let K > 1. Then for every e > one has 

(5.1) V(s n {A) < en- 1 ' 2 ) <Ce + c n + W{\\A\\ > Kn 1 ' 2 ), 

where C > and c G (0, 1) depend (polynomially) only on B and K. 
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This result implies the Subgaussian Invertibility Theorem 11.21 indeed, the 
last term in (15.11) is exponentially small by Lemma 12.41 

The imprecise term n~ 1//2 in the Weak Invertibility Theorem 13.11 came from 
from the Weak Distance Bound, Lemma 13.81 which estimated the distance 
between a random vector and a random hyperplane. Thus, in order to complete 
the proof of the Strong Invertibility Theorem 15.11 it suffices to improve the 
bound in Weak Distance Bound (Lemma 13.81) as follows: 

Theorem 5.2 (Strong Distance Bound). Let A be a random matrix as in The- 
orem \5.1\ Let X±, . . . , X n denote its column vectors, and consider the subspace 
H n = span(Xi, . . . , X n _x). Let K > 1. Then for every e > 0, one has 

P(distpf n , H n ) < e and \\A\\ < Kn 1/2 ) < C 7 (e + c n ), 

where C-j and c G (0, 1) depend only on B and K . 

Remark. For random vectors with independent ±1 coordinates, a weaker bound 
P(dist(X„, H n ) < -y < Clog" 1/2 n was proved by Tao and Vu [29]. 

5.1. Essential LCD of the random normal. As in Section [3.41 we shall 
estimate the distance by using the the random normal X*, a unit normal of 
the subspace H n . The inequality (13.81) reduces the problem to a lower bound 
on\(X*,X n )\. 

The random normal X* is convenient to control via the random matrix A', 
the (n-l)xn matrix with rows X 1; . . . , X n _i. Thus A' is the submatrix of A T 
obtained by removing the last row. By the definition of the random normal, 

A'X* = 0. 

We will use this observation as follows: 

(5.2) If II^L'xl^ > for all vectors x in some set S, then X* ^ S. 

Thus, a weak (qualitative) invertibility of the random matrix A' on S will help 
us to "navigate" the random normal X* away from undesired subsets 5* of the 
unit sphere. 

We shall use this approach to prove that the essential LCD of the random 
normal is exponentially large, with probability exponentially close to 1. This 
will allow us to use the full strength of the Small Ball Probability Theorem 11.51 
in order to bound \ (X*,X n )\ from below. 

Recall that x denotes the spread part of a vector x with some fixed truncation 
levels Ki,K 2 , see Definition 14.81 

Theorem 5.3 (Random normal). Let Xi, . . . be random vectors as in 

Theorem \5.2 . Consider a unit vector X* orthogonal to all these vectors. Let 
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K > 1. JTien there exist constants K\, K 2 , a, (3, c, d > that depend only on 
B and K , and such that 

F(D a>f3n (X*-) < e cn and \\A\\ < Kn 1/2 ) < e~ c ' n . 

Intuitively, the components of a random vector should be arithmetically 
incomparable to the extent that their essential LCD is exponential in n. In 
the case of the random normal X*, its components are not independent, and 
it requires some work to confirm this intuition. 

We shall prove that the random matrix A' is likely to be invertible on the 
subsets Sd of the unit sphere where the essential LCD is of order D, for each 
D below an exponential order. Then, by observation (I5.2p . the random normal 
X* will not lie in such Sd- Therefore, the essential LCD of X* will be at least 
of exponential order. 

5.2. The level sets of the essential LCD. Fix K > 1 for the rest of the 
proof. We shall first choose the truncation levels K\ = Ki(B,K), K 2 = 
K 2 (B, K) in the definition of the spread part X* of the random normal. 

Our soft invertibility argument in Section [XT] was based on considering sep- 
arately compressible and incompressible vectors, forming the sets Comp = 
Comp(5, p) and Incomp = In comp (5, p) respectively, see Definition 13.21 The 
parameters 5, p > in the definition of these vectors were chosen in Lemma [3731 
depending only on B and K. 

For every incompressible vector x, its spread part is proportionally large. 
Indeed, by Lemma 13.41 there exist Ki, K 2 , c > that depend only on B and 
K, and such that for the truncation levels Ki and K 2 one has supp(x) > Cqu. 
For the future convenience, we consider the even integer n := 2|_Con/2j . Thus 
we have: 

(5.3) Every x G Incomp satisfies |supp(x)| > n > ~^ n - 

We shall choose the value a £ (0, 1 /2) later. By the definition of the essential 
LCD and of the spread part, 

D Q , no/2 (x) > (1 - a)/K 2 > 1/2K 2 =: D . 

Definition 5.4 (Level sets of LCD). Let D > D . We define the level set 

S D C S 71 - 1 as 

Sd '■= {x G Incomp : D < D a ^ no / 2 (x) < 2D}. 

We want to show the invertibility of the random matrix A' on the level sets 
Sd for all D up to an exponential order. This will be done by a covering 
argument. We will first show the invertibility on a single vector x G Sd- 
Next, we will find a small (a/D)-net in Sd- Then, by a union bound, the 
invertibility will hold for each point in this net. By approximation, we will 
extend the invertibility to the whole Sd- 
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The invertibility on a single vector x G Sd will easily follow from our general 
small ball probability estimates and the Tensorization Lemma 12. 21 

Lemma 5.5 (Invertibility on a single vector). There exist c, Cg > that depend 
only on B and K , and such that the following holds. Let a G (0, 1) and 
Dq < D < ^e c " 2n . Then for every vector x G Sd and for every t > 0, one 
has 

V(\\A x \\ 2 <tn^)<(c,t+-^ V " 



Proof. Let £ki, . . . , £kn denote the fc-th row of A'. The k-th component of A'x 
is then (A'x)h = YTj=i x j 6y = '■ Ck- By Corollary 14.91 and by our assumption 
on D, for every k we have for all a G (0, 1): 

P(iai < t) < C(t + --) + Ce- ca2n ^ 2 <C'(t+ ' 



2 
fe) 



where C, c, C depend only on B and K. 

Since (±, . . . , are independent random variables and ||^4'a;||| = Ylk=i C 
Tensorization Lemma 12.21 with Eq = ^ D completes the proof. □ 

Remark. This proof only used the lower bound D a ^ m /2{x) > D in the definition 
of the level set Sd- 

Lemma 5.6 (Nets of the level sets). There exist «o G (0,1), Cg > and 
c 9 G (0, 1) that depend only on B and K , and such that the following holds. 
Let < a < ciq and D > Dq. Then there exists a (4a/ D) -net in Sd in the 
Euclidean metric, of cardinality at most 

C 9 D 



Remark. By a simple volumetric estimate (see e.g. [IE]), the sphere S 1 " -1 has 
an #-net of cardinality (3/9) n for every 9 > 0. This implies Lemma 15.61 with 
Cg = 0. The fact that the level sets have somewhat smaller cardinality, namely 
with eg > 0, will be crucial in our argument. 

Proof. We start by constructing a (2a/D)-net for Sd of the desired cardinality, 
whose elements do not necessarily belong to So- 
Let x £ Sd- Recall that supp(x) > n by (15. 3p . By the definition of 
D(x) = D aino / 2 (x), there exist q G M supp(x ) with no/2 integer coefficients and 
such that 

||D(x) x — q\\oo < a. 
We can extend q to a vector in M. n by quantizing its non-integer coefficients 
uniformly with with step a. Thus there exists pel" whose no/2 coefficients 
are in Z and whose other coefficients are in aL, and such that 

(5.4) || \fnD(x) x — plloo < a. 
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(Recall that x is a restriction of a vector y/rix). We thus have p G V, where 
(5.5) V := [J Z a ®aZ aC , 

W\=n /2 



the union being over all (no/2)-element subsets a of {1, 
It follows from ( 15.41) and Holder's inequality that 



n}. 



(5.6) 

Using i G Sd, we obtain 



| y/nD(x) x — p\\2 < ct\fn. 



x — 



P 



a a an 1 
- D(x) ~ D ~ D ~ 4' 



y/nD(x 

if we choose a := min(l, D /4). 

Now we use the following elementary implication, which holds for every pair 
of vectors y and z in a Hilbert space: if ||y|| = 1 and \\y — z\\ < 5 < 1/4 then 
\\y — || < 25. This implies 



(5.7) 



x — 



P 



\Ph 



< 2a/D. 



On the other hand, since x is a unit vector, (15.61) implies 
IMIa < (D(x) + a)V^ < 3V^D 
where we used that a < a < D < D(x). We have thus shown that the set 

N = l-^r- -.pern 3^D ■ B%\ c R n 
l \\Ph > 

is a (2a/L>)-net for Sd- 

Let us estimate the cardinality of Af. There are ( n ™/ 2 ) < 2 n ways to choose 
the subset a in (15. 5j) . Then 



n— no/2 I 



< 2 n • \Z no/2 n 3v^£> • 52 0/2 | • |aZ n - n °/ 2 n 3t/ED ■ B, 

The Euclidean ball in M, d of radius R\fd and centered at the origin contains at 
most (CR) d integer points, where C is an absolute constant. Then, using that 
uq > cqti/2, we conclude that 

C 9 D 



\M\ < T ■ (C ■ 3D) no/2 ■ (C ■ 3D /a 



\n~no/2 



< 



a 



l-Cg 



Thus, M C W 1 is a (2a/D)-net for Sd of the required cardinality. To complete 
the proof, note that we can make M a subset of Sd using the following standard 
observation: 
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Lemma 5.7. Let T be a metric space and let E C T. Let Af C T be a 9 -net 
of the set E. Then there exists a {29) -net N' of E whose cardinality does not 
exceed that of Af ' , and such that Af' C E. 

□ 

Remark. As we see from (15.51) . we were able to construct a small net because 
of the coarse quantization of a coordinate subspace R CT of proportional dimen- 
sion, which we could afford due to the control of the essential LCD. The finer 
quantization of the complement M, aC , i.e. aZ^, can be replaced with an arbi- 
trary a-net of that subspace. The particular form of the net there does not 
matter. 

Lemma 5.8 (Invertibility on a level set). There exist a, c, c±q > that depend 
only on B and K , and such that the following holds. Let Dq < D < e cn . Then 

P( inf \\A'x\\ 2 < — n 1 ' 2 and \\A\\ < Kn 1/2 ) < e~ n . 

Proof. Recall that we can assume that n is sufficiently large. We shall therefore 
choose a value of a from the non-empty interval (-^, olq). Assume that D < 

D < -4= e c ° 2n as in Lemma 15.51 

We apply Lemma 15.51 with t = 5Ka/D; thus the term C%t will dominate 
over the term C%/y/nD. We therefore obtain for each x £ Sd'- 

Let Af be a (4a/D)-net of Sd constructed in Lemma 15.61 Then taking the 
union bound, we obtain 

Using the assumption D < e cn , we conclude that 

(5.8) P( inf WA'xoh < — n 1/2 ) < (C'aY^- 1 < e~ n , 

\x £Af D J 

provided that we choose at > appropriately small in the interval 
depending only on C' g and eg, which in turn depend only on B and K. 

We are now ready to bound the event V that ||A|| < Kn 1 ^ 2 and for some 
x E Sd, \\A'x\\2 < 7f n 1 / 2 . Assume that V occurs, and choose x$ E Af so that 
11^ — ^olh < Aa/D. Since ||A'|| < ||A|| < Kn 1 ^ 2 , we have 

WA'xoh < \\A'x\\ 2 + \\A'\\\\x-xo\\ 2 < ^n^ + Kn 1 / 2 - < ^-n 1 ^, 

if we choose Co := Ka (which thus depends only on B and K). By (I5.8p . this 
completes the proof. □ 
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5.3. Proof of the Random Normal Theorem. Now we prove Theorem l5.3l 
Let a and c be as in Lemma 15.81 

If x G S'" 1 ^ 1 is such that D(x) < e cn then, by the definition of the level sets 
Sd, either x is compressible or x G So for some D G V, where 

V = {D: D /2 <D< e cn , D = 2 k , k G Z}. 

Therefore, denoting the event that \\A\\ < Kn 1 ^ 2 by Uk, we have 

¥(D(X*) < e cn and U K ) < P(X* G Comp and U K ) + J2 P ( X * G 5 « and £/*). 

Dev 

By Lemma [376l P(X* G Comp and £7^-) < e~ C4 ™. By (15. 2p and Lemma [5751 for 
every D G P we have 

P(X* G S D and E^) < P( inf \\A'x\\ 2 = and U K ) < e~ n . 

Since \T>\ < C'n, we conclude that 

F(D(X*) < e cn and U K ) < e~ C4n + C'n ■ e~ n < e~ c ' n . 
This completes the proof of Theorem 15.31 □ 

5.4. Proof of the Strong Distance Bound and the Strong Invertibility 
Theorem. Now we deduce Theorem 15 . 21 from our small ball probability bound 
(Corollary 14. 9[) and the Random Normal Theorem 15.31 

We proceed with a conditioning argument similar to those used to prove 
the Weak Distance Bound, Lemma 13. 81 We condition upon a realization of 
the random vectors Xi, . . . , X n _i. This fixes realizations of the subspace H n 
and the random normal X*. Recall that X n is independent of X*. We denote 
the probability with respect to X n by P n , and the expectation with respect to 
Xt, . . . , X n _i by Ei,... >n _i. Then 

F(\(X*,X n )\ < e and ||A|| < Kn l/2 ) 

< E 1 ,..., n _ 1 P n (|(X*,X n )| < e and D a ^ n (X*) > e cn ) 

+ F(D a)l3n (X*) < e cn and \\A\\ < Kn 1/2 ). 

By the Random Normal Theorem I5.3[ the last term in the right hand side 
is bounded by e~ cn . Furthermore, by Corollary 14.91 for any fixed realization 
of Xi, . . . , X n such that D a ^ n (X*) > e cn we have 

P n (|(X*,X n )| <e) < C"e + C"e- C " n . 

It follows that 

P(|(X*,X n )| < e and \\A\\ < Kn 1/2 ) < C"e + C"e~ c " n + e~ c ' n . 
By (I3.8p . the proof of Theorem 15.21 is complete. □ 
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Combining Lemma 13.51 and Theorem 15.21 we deduce a strong invertibility 
bound for a random matrix on the set of incompressible vectors. This improves 
a polynomial term in Lemma 13.91 to an exponential term: 

Lemma 5.9 (Strong invertibility for incompressible vectors). Let A be a ran- 
dom matrix as in Theorem \5. 11 Let K > 1 and 5, p G (0, 1). Then for every 
e > 0, one has 

P( inf ||Ar|| 2 <epn~ 1/2 ) < ^-(e + c n ) +F(\\A\\ > Kn l/2 ), 

x£lncomp(S,p) 

where C\\ > and c G (0, 1) depend only on B and K . 

The Strong Invertibility Theorem 15.11 now follows from the decomposition 
of the sphere (13.21) into compressible and incompressible vectors, and from the 
invertibility on each of the two parts established in Lemma 13731 (see the remark 
below it) and Lemma 15.91 (used for 5, p as in Lemma 13.31 and for e/p rather 
than e). □ 
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